c O MPUTER TECHNOLOGY 

A New Continent of Ideas 


hat happens if an astronaut 
on a space station of tomor- 
row needs an emergency ap- 
pendectomy — and there is no 
surgeon on board? 

Telesurgery is one possibility — surgery per- 
formed by a robot whose movements are pre- 
cisely guided by a surgeon on Earth. He con- 
ducts the operation by a combination of com- 
puters, television and advanced sensors. The 
stereoscopic view entirely surrounds the doc- 
tor so that he feels he is actually a part of the 



space station scene, and he is able, through in- 
strumented glove technology, to direct the ro- 
bot's hand movements to correspond exactly to 
his own hand movements. 

Way out? Not really, according to scientists at 
Ames Research Center's Aerospace Human 
Factors Research Division. Telesurgery is not 
now available, of course, but it is considered 
feasible for a 2 1 st century time frame, say two to 
three decades from now. And it is only one of an 
infinite number of exciting potential applica- 
tions for a burgeoning new technology called 
“virtual reality.” 

The technology is still at the “ground floor” 
level, still somewhat crude, requiring a great 
deal of development and refinement. But one of 
its basic components— 3D computer graphics— 
is already in wide commercial use and expand- 
ing explosively. Other components that permit 
a human operator to “virtually” explore an arti- 
ficial environment and to interact with it are 
being demonstrated routinely at Ames and else- 
where. Some of them, in fact, are already com- 
mercially available, albeit expensively, and the 
technology developed for one of Ames' artificial 
reality research tools — the instrumented glove — 
has even found its way into a video game . 

Virtual reality (VR) might be defined as an 
environment capable of being virtually en- 
tered — telepresence, it is called — or interacted 
with by a human. One reason for NASA's interest 
is an anticipated need for large scale remote 


| 

control of robotic space systems. Robots are | 

becoming more and more sophisticated, more | 

and more dextrous, and there is need for corol- ,• 

lary development of devices, displays, skills and ' 

techniques that will allow telepresence and ef- 
fective telerobotic control. 

Since the mid-1980s, Ames' Aerospace Human 
Factors Research Division has been developing i; 

experimental systems that permit human/ com- 
puter interaction. For example, the VIEW sys- [ 

tern. VIEW stands for Virtual Interface Environ- - 

ment Workstation. It is a head-mounted stere- 
oscopic display system in which the display may 



Above, a NASA scientist is conducting a test of Ames 
Research Center's ” virtual reality ” headset. She sees a 
computer-generated 3D scene or a real environment 
remotely relayed by video cameras; the stereo imagery 
suggests that she is actually part of the scene. 
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Topping a selection of spinoffs in computer technology 
are hardware /software systems for advancing the 
exciting concept of virtual reality 


be an artificial computer-generated environment 
or a real environment relayed from remote video 
cameras. The operator can — virtually at least — 
“step into” this environment and interact with it. 

He wears a headset whose centerpiece is a 
display box containing two small (3.9-inch) tele- 
vision screens, one for each eye so that the TV 
image appears three dimensional. The scene is 
accompanied by appropriate sound effects de- 
livered to the headset. The headset bars view of 
anything but the imagery, thus helps create the 
illusion that the user is part of the scene pic- 
tured. One example of an important practical 
application: a design engineer can virtually 
become part of the fuel flow of a rocket engine 
and travel with the flow, noting places where it 
slows, speeds up or becomes turbulent; he can 
learn a great deal about system design that he 
could not in 2D simulation. 

The scene might be a room. If the operator 
turns his head, the scene shifts just as it would 
in the real world; a headset-mounted sensor 
tracks the position of the user’s head and com- 
municates this knowledge to the computer. By 
pointing in a given direction, the operator virtu- 
ally moves in that direction so he can explore 
any part of the room. Ames is developing a 
library of software for various scenes and the 
operator can select a menu option by a word or 
gesture, because the system is trained to recog- 
nize voice and gesture commands. 

An important addition to the Ames system is 
the DataGlove™ , an experiment in telepresence 
control of advanced robotic hands or fingers. 
Developed for Ames by VPL Research, Inc., 
Redwood City, California, the glove has a series 
of fiber optic cables and sensors that detect any 
movement of the wearer's fingers and transmit 
the information to a host computer; a computer- 
generated image of the hand will move exactly as 
the operator is moving his gloved hand. With 
appropriate software, the operator can use the 
glove to interact with the computer scene by 
grasping an object, for example, moving a virtual 
chair within the simulated room; the computer 


will dutifully move the chair in the TV display. 
Not only that, the operator can “feel” the virtual 
chair through tiny vibrators in the fingertips of 
the DataGlove. 

It is possible, Ames and other VR researchers 
feel, to replicate almost any environment or 
activity, so VR has immense potential, both as a 
research tool and an operational system for 
telepresence/telerobotics. 

The possibilities for practical applications of 
VR in everyday life are even broader, thinks 
Jaron Z. Lanier. Lanier is chief executive officer 
of VPL Research, developer of the DataGlove 
and other systems for creating virtual reality, 
which he describes as “a new continent of ideas.” 

(Continued) 



A possible future application of virtual reality: an air 
traffic controller wearing a helmeted version of the 
headset monitors aircraft from a virtual viewpoint in 
the airspace. 


™ DataGlove is a trademark of VPL Research, Inc. 
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Jaron Lanier, 29, originally planned to be a 
professional musician, got sidetracked into 
computers and software, then into designing 
video games. In 1984, he met Tom Zimmerman, 
who was developing a wired glove with which to 
play an imaginary “air guitar.” Lanier recognized 
the virtual reality potential of Zimmerman’s 
concept, persuaded Zimmerman to join him and 
founded VPL Research to pursue the glove idea 
and related VR systems. 

In April 1986, VPL won a contract for its first 
product, the NASA DataGlove. Made of thin 
Lycra, the gloves have sensors at 15 points that 
monitor flexion and extension of the fingers, the 
position and orientation of the hand and thus 
permit hand/computer interaction. The first 
glove was delivered to NASA in May 1986. 

During the NASA contract, Lanier and VPL 
scientists worked closely with NASA virtual 
reality principal investigator Scott S. Fisher and 
his telepresence/telerobotics researchers at 
Ames' Aerospace Human Factors Research Divi- 
sion. It was a mutually beneficial arrangement; 
NASA gained an important research capability 
with development of the DataGlove and VPL 
learned from extensive use of the technology 
incorporated in Ames’ VIEW system. 

In a remarkably short span, VPL has expanded 
its research and product line. A natural follow- 
on to the DataGlove was the DataSuit™, a sen- 
sor equipped full body garment that greatly in- 
creases the sphere of performance for virtual 
reality simulations by reporting to the computer 
the motions, bends, gestures and spatial orien- 
tation of the wearer— in other words, it makes 
possible full body interaction with the computer 
generated virtual world. The absolute position 
sensors for the DataGlove and DataSuit — and 
also for the head tracker in Ames’ VIEW sys- 
tem — are provided by Polhemus Inc., Colch- 
ester, Vermont. The DataSuit has already found 
a commercial application, an unusual one: it is 
worn by film actors to give fluid, realistic motion 
to animated characters in computer-generated 
movie special effects. 



At top is the DataGlove developed for NASA by VPL 
Research, a computer input device for manipulating 
3D environments. In the lower photo the movements 
of the gloved hand are duplicated on the screen by a 
computer-generated graphic hand. 
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After DataSuit, VPL created its own version of 
the eye display in NASA's helmet system, the 
EyePhone™ , a head-mounted stereo display. 
VPL has developed a line of software for virtual 
reality applications. The company offers a 
complete package, the RB2 Virtual Environment, 
that includes one DataGlove, the EyePhone, a 
design control workstation with processor and 
associated software, cables, transceivers and 
connections, for $45,000. With the computers, 
which are made by Silicon Graphics, Mountain 
View, California, the whole system goes for 
$200,000 for a single user, $400,000 for two users. 

That's costly, to be sure, but VR enthusiasts 
are counting on continuance of the dramatic 
drop in computer costs that made personal 
computers available to all and may similarly 
broaden VR applications. 

The DataGlove alone sells for about $9,000. 
Among immediate applications are Computer- 
Aided Design and Manufacturing (CAD-CAM), 
wherein the glove serves as an input device for 
manipulating 3D environments; the user can 
grasp objects on the screen to change an envi- 
ronment without lengthy keyboard stroking. In 
robotics, robot arms and grippers can be ma- 
nipulated in real time by the gloved hand, for 
training or telemanipulation. In animation, 
computer-generated characters can be pup- 
peted by mapping DataGlove sensor values to 
the characters' limbs or actions. 

The glove has medical applications, too, for 
example, it can be used for sign language re- 
search or as a tool for assessing hand function 
and performance; these applications are being 
developed by Greenleaf Medical Systems, Palo 
Alto, California. And, of course, it can be used to 


Wearing sensor-equipped DataGloves and a full-body 
DataSuit, a technician can virtually become part of a 
computer-generated environment. Sensor signals are 
picked up by the computer and the suit-wearer's 
motions, bends and gestures are replicated in the 
artificial environment. 


The VPL Research EyePhone is a headset in which the 
viewer can 't see the physical world, instead sees a 
computer generated VR world in color and 3D. With 
the DataGlove, the goggled VPL engineer shown can 
virtually grasp and pick up objects in the VR world as 
if they were real. 


interact with simulated environments for a va- 
riety of purposes; already on the market is Mat- 
tel Toys' simplified version of the glove for use 
in Nintendo video games. 

With improvements contemplated for the next 
few years, VPL sees such additional uses for the 
RB2 system as allowing an architect's clients to 
inspect and perhaps alter a building design be- 
fore the structure is built by virtually walking 
through a graphic replication of it; similarly, the 
system makes it possible to walk through dy- 
namic models of communication networks, large 
databases and traffic control systems. VR offers 
three-dimensional scientific visualization, par- 
ticularly useful in chemistry, geology and aero- 
dynamics. In medicine, it offers an additional 
increment of information-gathering capability 
when used as an image enhancement tool in 
diagnostic body imaging. 

VR has broad potential for training in a num- 
ber of fields. As for education, it permits virtual 
travel in time; a student can virtually be a phar- 
aoh in ancient Egypt, a tyrannosaurus hobnob- 
bing with other dinosaurs or a comet soaring 
past planets and moons and asteroids at a mil- 
lion miles an hour. And entertainment? Says 
VPL: “We leave that to your imagination. . .” 

“VR is shared and objectively present like the 
physical world,” company literature states, 
“composable as a work of art and as unlimited 
and harmless as a dream. When VR becomes 
widely available, it will not be seen as a medium 
used within a physical reality, but rather as an 
additional reality. VR opens up a new continent 
of ideas and possibilities.” 

™ DataSuit and EyePhone are trademarks of VPL 
Research, Inc. 
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